Classifying Very-High-Dimensional Data with Random Forests of Oblique Decision Trees

The random forests method is one of the most successful ensemble methods. However, random forests do not have high performance when dealing with very-high-dimensional data in presence of dependencies. In this case one can expect that there exist many combinations between the variables and unfortunately the usual random forests method does not effectively exploit this situation. We here investigate a new approach for supervised classification with a huge number of numerical attributes. We propose a random oblique decision trees method. It consists of randomly choosing a subset of predictive attributes and it uses SVM as a split function of these attributes. We compare, on 25 datasets, the effectiveness with classical measures (e.g. precision, recall, F1-measure and accuracy) of random forests of random oblique decision trees with SVMs and random forests of C4.5. Our proposal has significant better performance on very-high-dimensional datasets with slightly better results on lower dimensional datasets. Thanh-Nghi Do Institut Telecom; Telecom Bretagne UMR CNRS 3192 Lab-STICC Université européenne de Bretagne, France Can Tho University, Vietnam e-mail: [email protected] Philippe Lenca Institut Telecom; Telecom Bretagne UMR CNRS 3192 Lab-STICC Université européenne de Bretagne, France e-mail: [email protected] Stéphane Lallich Université de Lyon, Laboratoire ERIC, Lyon 2, France e-mail: [email protected] Nguyen-Khang Pham IRISA, Rennes, France Can Tho University, Vietnam e-mail: [email protected]